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ABSTRACT:  The  JJ.S.  Navy  has  established  a  Probability  of  Raid  Annihilation  (Pra)  Assessment  Process  to  be  used 
for  each  new  ship  class.  Papers  presented  at  previous  SIW  meetings  have  described  the  Pra  and  the  development  of 
certain  aspects  of  the  Pra  Assessment  Process,  including  the  Pra  Federation  Testbed.  This  paper  provides  an  overview 
of  the  total  Pra  Assessment  Process  to  illustrate  how  the  process  provides  the  Pra  ship  class  results  to  meet  OPEVAL 
requirements  across  ship  classes  in  a  consistent  and  adequate  manner. 

The  Fra  Assessment  Process  as  applied  to  each  new  ship  class  will  build  on  the  products  and  i esults,  including  the  Pra 
Federation  Testbed  as  implemented,  from  previous  ship  class  work.  To  facilitate  this  reuse,  all  relevant  material,  col¬ 
lectively  referred  to  as  the  Pra  Assessment  Process  Standards  and  Architecture  (PS&A),  is  documented  in  one  module. 
The  PS&A  module  is  the  roadmap  for  any  new  ship  class’s  program  manager  and  technical  team  to  implement  the  Pra 
Assessment  Process,  including  documentation  and  products. 

The  PS&A  shows  how  the  steps  in  the  Pra  Assessment  Process  correlate  to  the  steps  in  the  FEdercition  Development  & 
Execution  Process  (FEDEP)  model:  requirements,  conceptual  modeling,  design,  software  development,  integration,  and 
execution.  W&A  is  not  a  separate  step  but  overlays  all  of  the  steps  in  the  FEDEP  and  in  the  Pra  Assessment  Process. 
The  PS&A  includes  the  documents  and  products  associated  with  each  FEDEP  step,  including  the  Systems  Engineering 
Concept  Model  (SECM).  The  discussion  with  the  SECM  captures  both  the  real  world  and  simulated  views.  In  addition 
the  SECM  starts  with  a  generic  conceptual  view  and  then  granulates  to  specific  applications.  This  approach  takes  ad¬ 
vantage  of  what  is  common  among  the  ship  classes  while  capturing  the  critical  differences  as  they  relate  to  the  ability  of 
a  single  ship  to  defend  itself  against  a  threat  raid. 

1.  Introduction  defend  itself  against  multiple  anti-ship  cruise  missile 

threats  (a  raid).  The  Navy  has  determined  that  the  PRA 
The  Probability  of  Raid  Annihilation  (PRA)  Measure  of  M0E  must  be  assessed  for  each  new  ship  class.  As  part  of 
Effectiveness  (MOE)  is  the  measure  of  a  single  ship  to  a  risE  mitigation  strategy,  the  PRA  Assessment  Process 
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Standards  &  Architecture  (PS&A)  has  been  developed  to 
meet  OPEVAL  requirements  in  a  consistent  and  adequate 
manner.  The  PS&A  is  designed  to  reduce  costs  by  using 
standard  practices  and  tools  and  by  building  on  previous 
ship  class  work.  The  PS&A  provides  an  approved  road¬ 
map  for  each  ship  class’s  program  manager  and  technical 
team  to  assess  the  PRA  MOE  for  that  ship  class. 

The  PRA  MOE  is  difficult  to  assess  with  traditional  testing 
that  is  done  both  at  sea  and  in  laboratories.  Although  cost 
is  a  limiting  factor,  a  more  important  consideration  is  the 
inability  to  provide  complete  scenarios  that  include  all 
relevant  military  systems,  ship  and  threat,  to  test  their 
behaviors  and  interactions  during  the  “detect,  control  and 
engage”  timeline  for  ship  self  defense.  Therefore  the 
Navy  has  developed  the  PRA  Assessment  Process  to  in¬ 
clude  interoperable  simulations  together  with  a  robust 
validation  process  that  incorporates  all  available  results 
from  sea-based  and  land-based  testing  trials.  Further  de¬ 
tails  on  PRA  can  be  found  in  earlier  papers.  [7],  [10] 

The  PRA  Assessment  PS&A  is  based  on  the  Program  Ex¬ 
ecutive  Office  (PEO)  Integrated  Warfare  Systems  (IWS) 
Systems  Engineering  Process,  which  does  address  Mod¬ 
eling  &  Simulation.  The  PEO  IWS  Systems  Engineering 
Process  [9]  has  been  developed  using  standard  systems 
engineering  techniques  and  tools.  The  High  Level  Archi¬ 
tecture  (HLA)  FEderation  Development  and  Execution 
Process  (FEDEP)  incorporate  system  engineering  meth¬ 
ods  tailored  to  federation  development.  [5]  This  paper 
will  describe  the  PRA  Assessment  PS&A  and  its  relation¬ 
ship  to  the  FEDEP,  including  the  relation  of  the  Systems 
Engineering  Concept  Model  (SECM)  to  the  Federation 
Concept  Model.  Then  the  SECM  steps  as  applied  to  the 
Pra  Assessment  PS&A  are  described  in  detail. 

2.  Systems  Engineering 

Systems  engineering  as  a  specialty  has  been  established  to 
guide  the  engineering  of  a  complex  system  or  family  of 
systems.  The  process  is  not  restricted  to  just  hardware  or 
manufacturing  but  in  fact  covers  any  program  or  project 
from  its  inception  to  product  implementation  to  retirement 
of  such  products  by  outlining  in  general  terms  the  re¬ 
quired  steps  and  documentation  required.  Systems  engi¬ 
neering  can  be  considered  to  be  a  common  sense  ap¬ 
proach  to  development.  It  is  the  responsibility  of  the 
systems  engineer  to  employ  practices  that  are  of  sound 
judgment  from  cost,  schedule,  performance  and  risk  ob¬ 
jectives  including  the  needs  and  requirements  of  the  cus¬ 
tomers.  Systems  engineering  documentation  is  high  level 
and  refers  the  reader  to  accepted  standards,  guidelines  and 
IEEE  publications.  These  standards  have  been  developed 
to  be  rigorous  in  nature  for  continued  repeatability  and 
consistency  among  systems. 


2.1  The  Program  Executive  Office  (PEO)  Integrated 
Warfare  Systems  (IWS)  Systems  Engineering 
Process 

The  Program  Executive  Office  (PEO)  Integrated  Warfare 
Systems  (IWS)  Systems  Engineering  Process  [9]  ad¬ 
dresses  the  performance  baseline,  functional  baseline, 
allocated  baseline  and  product  baseline  for  a  system  or  in 
some  cases  a  family  of  systems.  The  documentation  is 
high  level  and  refers  the  reader  to  DoD  standards,  guide¬ 
lines  and  IEEE  publications.  The  PEO  IWS  Systems  En¬ 
gineering  Process  can  be  shown  graphically  (Figure  1) 
and  includes  a  detailed  checklist.  The  purpose  of  the 
checklist  is  to  provide  the  program  manager  a  useful  tool 
to  ensure  that  the  key  systems  engineering  questions  have 
been  answered  as  early  in  the  program  as  possible.  By 
doing  this,  the  systems  engineer  is  able  to  use  the  attrib¬ 
utes  of  the  systems  engineering  process  to  the  greatest 
advantage,  thus  achieving  total  system  optimization  as 
opposed  to  applying  bigger  corrections  farther  along  the 
development  process.  The  checklist  also  serves  as  a  re¬ 
minder/aid  for  the  program  manager  and  the  program 
systems  engineer  in  meeting  the  technical  systems  engi¬ 
neering  and  performance  requirements  of  the  program. 

The  PEO  IWS  Systems  Engineering  Process  includes  the 
option  of  using  Modeling  &  Simulation  in  its  broader 
meaning  of  computer-aided  tools  and  techniques,  but 
M&S  is  an  aspect  of  the  overall  systems  engineering 
process. 

2.2  The  High  Level  Architecture  (HLA)  Federation 
Development  and  Execution  Process  (FEDEP) 

The  High  Level  Architecture  (HLA)  is  a  general  purpose 
architecture  that  supports  interoperability  and  reuse  across 
the  many  different  types  of  simulations  developed  and 
maintained  by  the  DoD.  The  HLA  Federation  Develop¬ 
ment  and  Execution  Process  (FEDEP)  describes  a  gener¬ 
alized  process  for  building  HLA  federations.  Both  the 
HLA  and  the  FEDEP  have  been  developed  under  the 
leadership  of  the  Defense  Modeling  and  Simulation  Of¬ 
fice  (DMSO).  [5]  The  FEDEP  does  not  replace  any  ex¬ 
isting  management  or  engineering  processes;  rather  it 
provides  a  high-level  framework  for  HLA  federation  con¬ 
struction  into  which  lower-level  development  practices 
native  to  each  individual  application  area  can  be  easily 
integrated.  The  FEDEP  defines  a  generic,  common  sense 
systems  engineering  methodology  for  the  HLA  that  can 
and  should  be  tailored  to  meet  the  needs  of  individual 
applications.  The  systems  engineering  formality  is  deter¬ 
mined  by  the  size  and  complexity  of  the  applications  be¬ 
ing  used.  The  FEDEP  also  includes  checklists  associated 
with  the  various  stages  of  federation  development,  from 
definition  of  federation  objectives  to  federation  integra¬ 
tion  and  execution.  The  checklists  track  particular  docu- 


ments,  products,  and  decisions  expected  to  result  from 
passage  through  a  given  process  step.  Theses  checklists 
are  similar  in  nature  to  the  checklists  used  for  the  PEO 
IWS  Systems  Engineering  Process. 

The  HLA  FEDEP  in  conjunction  with  its  checklist  are 
offered  to  the  community  as  a  framework  for  identifying 
and  addressing  general  issues  as  discussed  within  the  full 


context  of  end-to  end  process  model  for  the  development 
of  distributed  simulation  environments  that  fully  conform 
with  the  HLA  specifications.  The  FEDEP  is  instantiated 
as  a  six-step  process  that  can  be  implemented  in  many 
different  ways  depending  on  the  application.  Thus  it  fol¬ 
lows  that  the  time  and  effort  required  to  build  an  HLA 
federation  may  also  vary  significantly. 


Process  Input 

•  Customer  Needs/Objectives/ 
Requirements 

-  Missions 

-  Measures  of  Effectiveness 

-  Environments 

-  Constraints 

•  Technology  Base 

•  Output  Requirements  from  Prior 
Development  Effort 

•  Program  Decision  Requirements 

•  Requirements  Applied  Through 
Specifications  and  Standards 


Requirements  Analysis 

Analyze  Missions  &  Environments 
Identify  Functional  Requirements 
Define/Refine  Performance  &  Design 
Constraint  Requirements 


Requirements  Loop 


System  Analysis 
&  Control 
(Balance) 


Functional  Analysis/Allocation 

•  Decompose  to  Lower-Level  Functions 

•  Allocate  Performance  &  Other  Limiting  Requirements 
to  All  Functional  Levels 

•  Define/Refine  Functional  Interfaces  (Internal/External) 

•  Define/Refine/Integrate  Functional  Architecture 


Design  Loop 


•  Trade-Off  Studies 

•  Effectiveness  Analyses 

•  Risk  Management 

•  Configuration  Management 

•  Interface  Management 

•  Data  Management 

•  Performance  Measurement  , 

-SEMS 
-TPM 

-Technical  Reviews 


Synthesis 

•  Transfer  Architectures  (Functional  to  Physical) 

•  Define  Alternative  System  Concepts,  Configuration 
Items  &  System  Elements 

•  Select  Preferred  Product  &  Process  Solutions 

•  Define/Refine  Physical  Interfaces  (Internal/External) 


Related  Terms: 

Customer  =  Organizations  Responsible  for  Primary  Functions 
Primary  Functions  =  Development,  Production/Construction,  Verification, 
Deployment,  Operations,  Support,  Training,  Disposal 
Systems  Elements  =  Hardware,  Software,  Personnel,  Facilities,  Data,  Material, 
Services,  Techniques 


Process  Output 

•  Development  Level  Dependent 

-  Decision  Data  Base 

-  System/Configuration  Item 
Architecture 

-  Specifications  &  Baselines 


Figure  1.  The  PEO  IWS  Systems  Engineering  Process 


2.3  Pra  Assessment  PS&A  Compared  to  HLA  FEDEP 

The  PRA  Assessment  PS&A,  based  on  Figure  1,  can  be 
described  generally  in  six  steps  similar  to  those  of  the 
FEDEP.  [5]  The  comparison  is  as  follows: 

FEDEP  Step  1:  Define  federation  objectives. 

The  federation  user  and  federation  development  team 
define  and  agree  on  a  set  of  objectives  and  document  what 
must  be  accomplished  to  achieve  those  objectives. 

PS&A  Step  1:  Process  Input. 

The  customer  needs,  objectives  and  requirements  are  de¬ 
termined.  A  technology  base  is  determined,  program  deci¬ 
sion  requirements  are  made  and  requirements  are  applied 
through  specifications  and  standards. 

FEDEP  Step  2:  Develop  federation  conceptual  model 
(FCM). 


Based  on  the  characteristics  of  the  problem  space,  an  ap¬ 
propriate  representation  of  the  real  world  domain  is  de¬ 
veloped  along  with  federation  requirements  and  test 
evaluation  criteria. 

PS&A  Step  2:  Requirements  analysis/SECM 
Analyze  missions  and  environments,  identify  functional 
requirements,  define/refine  performance  and  design  con¬ 
straint  requirements. 

FEDEP  Step  3:  Design  federation. 

Federation  participants  (federates)  are  determined,  re¬ 
quired  functionalities  are  allocated  to  the  federates  and  the 
federation  development  plan  is  designed. 

PS&A  Step  3:  Functional  analysis/allocation. 
Decomposition  to  lower  level  functions  is  completed.  The 
allocation  of  performance  and  other  limiting  requirements 
to  all  functional  levels  is  done.  Functional  interfaces  both 


internal  and  external  are  defined  or  refined  as  needed,  as 
well  as  the  functional  architecture. 

FEDEP  Step  4:  Develop  federation. 

The  Federation  Object  Model  (FOM)  and  FED  files  are 
developed,  federate  agreements  on  consistent  databases/ 
algorithms  are  established,  and  modifications  to  federates 
are  implemented  (as  required). 

PS&A  Step  4:  Synthesis. 

Transfer  of  architectures  from  functional  to  physical.  Al¬ 
ternative  system  concepts  are  defined  along  with  configu¬ 
ration  items  and  system  elements. 

FEDEP  Step  5:  Integrate  and  test  federation. 

All  necessary  federation  implementation  activities  are 
performed,  and  testing  is  conducted  to  ensure  that 
interoperability  requirements  are  being  met. 

PS&A  Step  5:  System  analysis  &  control  (balance). 
Trade-off  studies  are  completed,  effectiveness  analyses 
and  risk  management  are  determined.  Configuration  man¬ 
agement  and  interface  management  and  data  management 
are  performed.  Performance  measurements  including  the 
Technical  Performance  Measures  (TPM)  and  Technical 
Reviews  are  executed. 

FEDEP  Step  6:  Execute  federation/prepare  results. 

The  federation  is  executed,  outputs  are  generated  and  re¬ 
sults  are  provided. 

PS&A  Step  6:  Process  Output. 

The  process  output  is  development  level  dependent.  A 
decision  data  base  is  developed,  system  configuration 
item  architectures  are  established  and  specifications  and 
baselines  are  established. 

The  primary  difference  between  the  PS&A  and  the 
FEDEP  is  that  the  PS&A  includes  a  federation  as  one  of 
the  tools  used  in  the  entire  program  while  the  FEDEP  is 
specific  to  federations.  A  second  difference  is  that  the 
PS&A  uses  the  conceptual  modeling  in  a  very  broad 
sense.  Although  the  SECM  is  shown  in  the  PS&A  step  2, 
it  is  used  throughout  the  entire  PRA  Assessment  Process  to 
capture,  link  and  display  as  much  of  the  information  as 
possible  for  analysis  and  ready  access. 

All  information,  documents  and  products  relating  to  the 
Pra  Assessment  PS&A  are  included  in  a  CD_ROM  for 
ready  access  and  review.  The  products  include  the  SECM, 
the  PRA  FOM  and  FED  files,  and  the  PRA  Federation 
Agreements. 

3.  Systems  Engineering  Concept  Model 
(SECM) 


The  SECM  methodology  is  general  and  can  be  applied  to 
many  different  types  of  programs,  not  just  to  those  in¬ 
volving  interoperable  simulations.  The  methodology  in¬ 
cludes  both  the  process  and  a  product.  The  process  is  gen¬ 
eral  and  can  be  applied  to  any  program  but  the  product  is 
specific  to  a  given  program  although  many  parts  of  a 
SECM  product  may  be  reused  in  related  programs.  The 
SECM,  as  the  name  implies,  involves  conceptual  model¬ 
ing.  In  general,  a  concept  model  views  the  world,  real  or 
simulated,  as  consisting  of  objects  with  defined  properties 
and  behaviors  that  interact  in  prescribed  ways.  [1]  The 
SECM  makes  extensive  use  of  graphical  representations 
that  use  a  precise  set  of  rules  or  language  in  capturing 
information  for  display.  [6]  The  SECM  also  captures  the 
detailed  analysis  performed  to  develop  the  federation  re¬ 
quirements;  therefore  it  can  serve  also  as  the  basis  for  the 
Verification  &  Validation  process  required  for  all  simula¬ 
tions.  [8],  [12] 

The  SECM  methodology  has  three  critical  factors  which 
must  be  considered  at  all  stages  of  development.  First,  a 
careful  identification  must  be  made  of  what  is  relevant 
and  what  is  not  relevant  to  the  program  objectives.  What 
is  relevant  will  shift  because  the  program  objectives  will 
be  modified  due  to  changes  in  resources,  priorities,  and 
stakeholders.  Second,  for  programs  involving  models  and 
simulations,  a  clear  separation  must  be  made  between  the 
real  world  view  and  the  simulated  world  view.  The  SECM 
product  or  electronic  document  provides  traceability  for 
all  decisions,  constraints,  approximations  and  assump¬ 
tions  leading  from  the  real  world  view  to  the  simulated 
world  view  as  well  as  capturing  the  complete  details  of 
each  view.  Third,  the  objects  in  the  military  systems  in¬ 
teract  through  and  with  the  natural  environment,  whether 
in  the  real  world  view  or  the  simulated  world  view. 
Therefore,  the  natural  environment  system  itself  should  be 
treated  as  consisting  of  objects  with  defined  properties 
and  behaviors  that  interact  in  prescribed  ways. 

The  main  steps  in  developing  the  SECM  for  the  PRA  As¬ 
sessment  Program  are  as  follows; 

1.  Develop  Use  Cases 

2.  Identify  Systems  Generic  (real  world  view  only) 

3.  Identify  Systems  Specific 

4.  Capture  System  Interactions  (may  be  done  be¬ 
fore  Step  3  as  well) 

5.  Develop  Scenario  Environment 

These  series  of  steps  are  done  twice;  first  for  the  real 
world  view  and  second  for  the  simulated  world  view 
based  on  what  has  been  captured  for  the  real  world  view. 
The  question  arises  as  to  how  the  data  from  the  testing 
trials,  both  sea-based  and  land-based,  fit  in.  To  date,  the 
data  is  captured  as  part  of  the  real  world  view  to  be  used 


in  the  VV&A  process  for  the  Pra  Federation.  However, 
for  future  work,  the  testing  trials  information  may  be  used 
to  develop  scenarios  for  the  Pra  Federation  runs  to  facili¬ 
tate  the  comparison  between  the  test  results  and  the  Pra 
Federation  results.  [11] 

The  importance  of  the  natural  environment  to  simulations 
and  the  use  of  concept  models  to  develop  that  natural  en¬ 
vironment  have  been  presented  in  previous  papers.  [3],  [4] 

3.1  The  Real  World  View 

The  <Real  World  System>  can  be  divided  into  three 
classes:  <Humans>,  <Natural  Environment  System>  and 
<Human-Made  Systemsx  Some  stakeholders  have  classi¬ 
fied  certain  human-made  structures,  such  as  non-military 
and  fixed  structures,  as  part  of  the  natural  environment. 
The  developers  of  the  SECM  procedure  have  found  that 
keeping  all  human-made  structures  in  the  <Human-Made 
Systems>  is  more  straight-forward  to  most  stakeholders. 
The  essential  point  is  that  all  stakeholders  must  agree  on 
which  classification  is  used.  (Figure  2.) 


Figure  2.  The  Real  World  System/View 


The  two  classes  <Humans>  and  <Human-Made  Systems> 
can  be  further  divided  into  Civil  and  Military  classes. 
Another  possible  division  is  into  Friend  and  Foe,  de¬ 
pending  on  the  scenarios  being  developed. 

3.1.1  Develop  Use  Cases  -  Real  World 

Strictly  speaking,  a  Use  Case  represents  one  completed 
set  of  actions  between  a  user  and  a  software  system. 
However,  the  term  has  evolved  into  more  general  usage 
including  that  of  scenario. 


The  first  step  in  detailing  the  Scenario  Use  Cases  is  to 
identity  the  objects  involved  in  the  PRA  scenarios.  As  seen 
in  Figure  3,  the  smallest  number  of  objects  or  classes  in¬ 
volved  in  the  scenario  is  three:  the  ship,  the  threat  and  the 
natural  environment.  <Humans>  are  not  included.  The 
number  of  threats  has  not  been  specified  at  this  point  but 
there  can  be  one  or  more  than  one.  The  number  of  ships  is 
limited  to  one  for  current  scenarios.  The  top  level  view 
shown  in  Figure  3  captures  these  classes  and  some  details 
of  their  relationships.  The  threat  and  the  ship  are  associ¬ 
ated.  The  role  of  the  threat  is  to  hit  the  ship  and  the  role  of 
the  ship  is  to  defend  the  ship.  The  numeral  1  to  the  left  of 
the  ship  class  indicates  that  one  ship  is  involved.  The  nu¬ 
merals  l...n  to  the  right  of  the  threat  class  indicates  that 
the  number  of  threats  can  be  1  to  n  where  n  is  agreed 
upon  by  the  stakeholders.  The  notation  within  the  class 
boxes  show  where  the  classes  are  defined  in  detail.  Both 
of  these  classes  are  associated  with  the  environment  class. 
Again,  the  notation  is  precise.  Only  one  environment  is 
involved.  Further,  the  direction  of  the  open  arrows  indi¬ 
cates  responsibility.  The  threat  class  has  the  responsibility 
to  tell  what  environment  it  uses  or  needs.  Similarly,  the 
ship  class  has  the  responsibility  to  tell  what  environment 
it  uses  or  needs.  And  the  environment  class  as  a  whole 
must  be  common.  Only  after  the  environment  class  in 
detail  is  determined  can  the  environment  class  in  turn  tell 
what  the  environment  database  should  be.  These  concepts 
of  classes  and  relationships  are  not  merely  graphical  but 
are  inherent  in  the  rigor  applied  to  the  way  in  which  the 
information  is  captured  in  the  electronic  document. 

The  object  classes  and  some  interactions  for  the  PRA  sce¬ 
nario  are  shown  in  Figure  3  but  the  Scenario  Use  Cases 
capture  the  events  in  the  scenario.  There  are  several  ways 
to  view  the  Scenario  Use  Cases.  One  which  focuses  on 
the  activities  in  the  PRA  scenario  is  shown  in  Figure  4.  The 
ship  behavior  includes  (detect,  control  and  engage)  the 
threat  while  the  threat  behavior  includes  (detect,  control 
and  engage)  the  ship.  However,  the  ship  behavior  itself 
and  the  activities  of  detecting,  controlling  and  engaging 
the  threat  all  use  the  environment  effects.  And  the  same  is 
true  for  the  threat  behavior  and  activities.  The  arrow  style 
used  here  shows  that  the  behaviors  and  activities  all  have 
the  responsibility  to  indicate  what  environmental  effects 
they  use. 

The  SECM  process  uses  as  many  views  as  necessary  to 
clarify  what  is  happening.  Not  all  views  can  show  all 
classes  and  interactions.  However,  once  links  between 
objects  are  established,  those  links  will  remain  whenever 
those  two  objects  are  used  again.  For  example,  any  view 
that  has  both  <Ship>  and  <Threat>  included  will  also 
include  the  link  shown  in  Figure  3. 


Figure  3.  Scenario  -  Top  Level  View 


Figure  4.  Use  Case  with  Behaviors  Identified  -  Real  World  View 


3.1.2  Identify  Military  Systems  -  General  -  Real 
World  View 

Once  the  Scenario  Use  Cases  have  been  defined  for  the 
Pra  scenario,  the  details  needed  for  the  various  military 
systems  are  needed.  For  example,  a  ship  consists  of  vari¬ 
ous  parts  or  components.  These  components  aggregate 
into  the  whole  that  is  called  the  ship.  For  the  PRA  scenario 
the  chief  classes  of  ship  components  are  the  sensors, 


weapons,  and  the  ship  platform  itself.  Obviously,  these 
are  not  the  only  components  of  a  ship.  However,  they  are 
the  minimum  relevant  set  for  this  PRA  scenario  as  agreed 
by  all  stakeholders.  At  this  point,  some  flexibility  is  pos¬ 
sible.  Each  of  the  three  ship  components  (classes)  can  be 
broken  down  into  further  components  if  desired.  For  con¬ 
ciseness,  only  the  breakdown  for  the  sensors  will  be 
shown. 


Figure  5.  Ship  Definition:  Sensors  (Partial  View)  -  Real  World  View 


Sensors  in  general  are  categorized  as  either  active  or  pas¬ 
sive.  Furthermore,  sensors  are  categorized  by  the  particu¬ 
lar  EM  frequencies  that  they  sense.  Thus,  it  is  logical  to 
define  classes  for  both  the  active  and  passive  sensors  in 
the  most  common  EM  ranges:  RF,  IR  and  Visible.  All  of 
these  classes  aggregate  into  the  class  labeled  Sensor  in 
Figure  5.  Again,  to  avoid  complicating  the  view,  only  the 
RF  Active  class  is  further  divided.  The  classes  of  RF  Ac¬ 
tive  Sensors  included  here  are:  Radar  and  IFF  (Identifica¬ 
tion  Friend  or  Foe).  RF  Active  Radars  have  several  differ¬ 
ent  functions  aboard  a  ship,  including  fire  control,  active 
search  and  surface  search.  Some  Attributes  of  the  Radar 
Class  are  shown  in  that  box  in  Figure  5.  An  RF  Active 
Radar  will  provide  target  range,  azimuth,  elevation  and 
track,  each  with  a  particular  accuracy,  depending  on  the 
specific  type  of  radar.  The  accuracies  are  detailed  as  At¬ 


tributes  because  the  PRA  MOE  will  itself  have  an  uncer¬ 
tainty  that  will  depend  on  the  accuracy  of  every  element 
involved  in  the  PRA  Scenario. 

3.1.3  Capture  System  Interactions 

The  sensors,  weapons,  and  the  ship  platform  interact  in 
various  ways.  These  interactions  must  also  be  captured. 
For  example,  the  ship’s  weapons  can  include  decoys 
which  alter  or  mask  the  ship  platform  signature.  The  part 
(frequency  range)  of  the  signature  that  is  altered  depends 
on  the  type  of  decoy.  This  type  of  interaction  is  shown  in 
Figure  6.  It  illustrates  that  the  entities  on  the  ship,  decoy 
and  platform,  have  a  relationship  that  involves  the  natural 
environment. 


Figure  6.  Ship  Definition:  Platform  (Partial  View)  -  Real  World  View 


The  discussion  to  this  point  has  been  intended  to  provide 
an  understanding  of  how  the  SECM  process  has  been 
used  to  develop  the  Pra  SECM.  The  Pra  SECM  itself  has 
many  more  views  and  details  of  the  Use  Cases  and  the 
military  systems  than  have  been  shown  here.  Further¬ 
more,  the  concept  model  development  to  this  point  has 
been  very  general.  Ship  class  has  not  been  specified  nor 
have  the  specific  weapons  or  sensors.  This  approach  is 
intentional  to  permit  the  PRA  SECM  to  be  reused  for  a 
wide  range  of  ships,  weapons  and  sensors.  Furthermore, 
Figure  5  does  not  show  all  of  the  environmental  effects 
that  will  be  used  by  the  various  classes  depicted.  Captur¬ 
ing  all  of  the  relevant  environmental  effects  and  illustrat¬ 
ing  them  at  this  general  level  is  very  complex.  A  Standard 


Environment  Template  for  each  Military  System  Class  is 
under  development  to  simplify  both  the  capture  and  dis¬ 
play  at  this  general  level. 

3.1.4  Identify  Military  Systems  -  Specific  -  Real 
World  View 

The  next  step  in  the  PRA  SECM  development  is  to  specify 
the  military  systems  being  considered  in  a  particular  PRA 
scenario.  One  scenario  may  have  three  specific  sensors 
for  the  ship:  SLQ-32,  SPQ-9B,  and  SPS-49  with  the  RAM 
Block  0/1  being  the  weapon  for  the  hard  kill.  The  ship 
may  be  the  LPD-17.  These  are  shown  in  Figure  7. 
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Figure  7.  Ship  Definition:  Military  Systems  -  Specific 
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Once  the  specific  ship  systems  are  identified,  all  interac¬ 
tions  among  the  ship  systems  must  be  identified,  includ¬ 
ing  what  type  of  information  is  passed  among  the  sys¬ 
tems.  Further,  the  natural  environment  parameters  that 
affect  the  behavior,  performance  and  interactions  for  each 
ship  system  must  be  captured  as  well.  The  Standard  Envi¬ 


ronment  Templates  are  a  useful  starting  point  but  must  be 
examined  for  each  specific  military  system.  Each  new 
system  generally  has  differences  behavior,  performance 
and  interactions,  especially  with  the  natural  environment 
than  the  existing  systems  of  that  class. 


3.1.5  Develop  Scenario  Environment 

The  analysis  done  for  the  ship  has  to  be  done  for  the 
threat  as  well.  Further,  the  interactions  between  the  ship 
and  threat  have  to  be  captured  as  well  as  between  the 
threat  and  the  natural  environment.  Once  all  parameters  of 
the  natural  environment  have  been  captured  for  each  spe¬ 
cific  military  system  and  for  all  interactions,  the  informa¬ 
tion  can  be  combined  and  analyzed  for  the  total  inferred 
natural  environment.  The  term  ‘inferred’  is  used  because 
the  parameters  must  be  inferred  from  the  behavior  and 
interactions  of  the  military  systems  in  the  real  world  sce¬ 
nario.  The  parameters  cannot  be  determined  before  the 
complete  analysis  is  done.  See  Figure  8. 


Two  classes  included  in  Figure  8  generally  are  not  con¬ 
sidered  to  be  natural  environment  effects:  location  and 
orientation.  The  Pra  developers  have  determined  that  the 
location  and  orientation  for  each  and  every  military  sys¬ 
tem  play  a  vital  role  in  assessing  the  Pra  MOE.  Further, 
the  natural  environment  data  values  that  affect  each  mili¬ 
tary  system  will  be  functions  of  location  and,  sometimes, 
orientation  as  well.  Therefore,  to  ensure  location  and  ori¬ 
entation  information  is  captured  and  linked  for  all  military 
systems,  these  classes  are  included  as  part  of  the  inferred 
natural  environment  system. 
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Figure  8.  Inferred  (Real  World  View)  Natural  Environment  for  the  Pra  Assessment  Process 


3.2  Simulated  World  View 

3.2.1  Develop  Use  Cases  -  Simulated  World  View 

At  this  point,  the  analysis  turns  finally  to  the  simulated 
world  view,  or  what  is  in  the  actual  PRA  Federation.  The 
first  step  is  to  develop  the  Use  Cases.  Here,  the  term  Use 
Cases  refers  to  the  specific  events  that  comprise  one  pass 
through  the  PRA  Federation,  not  what  happens  in  the  real 
world  view.  The  Use  Case  to  Assess  the  PRA  MOE  with 


the  Pra  Federation  is  shown  in  Figure  9.  This  Use  Case 
Diagram  does  not  contain  all  details  but  it  is  indicative  of 
the  differences  between  the  Use  Cases  for  the  real  world 
view  and  those  for  the  simulated  world  view. 

As  noted  in  Section  3,  the  Step  2  “Identify  Systems  Ge¬ 
neric”  is  omitted  for  the  simulated  world  view.  The  reason 
is  that  in  the  simulated  world  view,  the  classes  captured 
represent  models  of  the  real  world  systems,  not  the  sys¬ 
tems  themselves,  and  generic  models  are  rarely  available 


and  not  relevant  to  the  Pra  program.  Thus,  the  next  step  is 
to  identify  the  military  systems  specific.  This  view,  not 
shown  here,  appears  to  be  identical  to  that  shown  in  Fig¬ 
ure  7  except  for  a  label  of  Simulated  World  in  place  of 
Real  World.  However,  the  classes  shown  are  models  of 
the  specific  military  systems,  not  the  military  systems 
themselves.  Therefore  the  SECM  process  must  capture 
the  attributes  and  behaviors  of  the  models  themselves. 
The  model  attributes  and  behaviors  will  be  primarily  a 
subset  of  those  of  the  real  world  military  system.  How¬ 
ever,  the  models  may  have  additional  attributes  and  be¬ 
haviors  not  found  in  the  real  world  system.  For  example, 
some  models  contain  natural  environment  parameters  and 
databases.  Two  models  may  use  the  same  natural  envi¬ 
ronment  parameters  internally  but  have  different  values 
for  these  parameters. 


3.2.2  Capture  System  Interactions  -  Simulated  View 

The  system  interactions  for  the  simulated  view  depend  not 
only  on  model  attributes  and  behaviors  but  how  these 
models  are  linked  into  the  Pra  Federation.  A  simplified 
view  of  the  PRA  Federation  is  shown  in  Figure  10.  The 
interactions  for  this  PRA  Federation  are  determined  by 
tracking  information  into  and  out  of  each  model  and  each 
federate.  The  interactions  occur  across  the  HLA  RTI  but 
also  within  each  federate.  Many  of  these  models  and 
federates  are  legacy  systems  and  may  contain  algorithms 
and  databases  that  are  inconsistent  with  those  found  in 
other  models  and  federates.  All  of  this  information  must 
be  captured  and  analyzed  in  order  to  evaluate  the  merit  of 
the  Pra  MOE  that  is  obtained  from  the  PRA  Federation 
runs. 


Figure  9.  Use  Case  -  Simulated  World  View 


To  demonstrate  how  different  algorithms  can  cause 
problems,  consider  location  and  orientation.  Many  of 


the  military  system  models  use  a  body-centric  coordi¬ 
nate  system.  The  origin  of  that  coordinate  system  must 


be  identified  as  well  as  the  type  of  coordinate  system. 
Most  body-centric  coordinate  systems  are  Cartesian  (X, 
Y,  Z)  but  the  Z  axis  may  be  “up”  relative  to  the  body  or 
“down”.  As  information  is  passed  into  and  out  of  a 


model  or  federate,  coordinate  system  transformations 
have  to  be  made,  even  if  the  stakeholders  agree  that 
there  is  only  one  coordinate  system  to  be  used  for  in¬ 
formation  passed  across  the  RTI. 
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Figure  10.  The  Pra  Federation  -  Simulated  World  View 


3.2.3  Develop  Scenario  Environment  -  Simulated 
World  View 

The  SECM  captures  what  natural  environment  informa¬ 
tion  moves  into  and  out  of  each  model  and  federate  and 
how  that  information  is  added  to  or  modified  and  applied 
within  each  model  and  federate.  Then  all  of  these  cap¬ 
tured  aspects  of  the  natural  environment  are  analyzed  to 
identify  what  the  common  and  consistent  natural  envi¬ 
ronment  should  be  for  the  PRA  Federation  as  a  whole.  The 
resulting  list  of  parameters  should  be  a  subset  of  what  is 
in  the  inferred  natural  environment.  (Figure  8)  Otherwise, 
there  is  an  inconsistency  between  the  real  world  and  the 
simulated  world  that  will  impair  the  validity  of  the  fed¬ 
eration  results.  The  implemented  natural  environment 
includes  not  only  the  identified  parameters,  but  the  appro¬ 
priate  spatial  and  temporal  resolution  for  each  parameter. 
The  scenario/environment  generator  (Figure  10)  supplies 
the  implemented  natural  environment  data  for  each  PRA 
federation  run.  Each  model  and  federate  must  be  set  to 


receive  and  implement  the  natural  environment  data  sup¬ 
plied  by  the  scenario  generator  to  insure  a  “level  playing 
field”. 

4.  Reuse 

The  PRA  Federation  will  be  modified  for  each  ship  class  as 
the  specific  ship  military  systems  are  changed  in  the  real 
world  view.  The  PRA  Federation  will  be  modified  as  the 
Federation  itself  is  changed.  Further  the  threat  shown 
above  has  not  been  specified.  A  single  ship  class  may  be 
tested  against  several  types  of  threats  in  determining  the 
PRA  MOE.  Each  of  these  changes  must  be  captured  in  the 
SECM.  However,  large  portions  of  the  SECM  can  be  re¬ 
used.  For  example,  once  the  SPQ-9B  model  has  been 
analyzed,  that  analysis  can  be  reused  until  the  SPQ-9B 
model  is  modified.  And  an  additional  sensor  or  weapons 
can  be  added  to  the  existing  ones  in  the  real  world  and 
then  linking  that  sensor  or  weapon  model  to  the  existing 
federation.  The  entire  simulated  world  view  does  not  have 


to  be  created  from  the  beginning.  Only  the  attributes,  be¬ 
haviors  and  interaction  of  the  new  sensor  or  weapon 
model  needs  to  be  incorporated.  The  sharing  and  reuse  of 
concept  models,  especially  in  relation  to  developing  the 
natural  environment  requirements,  has  been  addressed  in 
a  previous  paper  [2] 

5.  Summary 

The  PRA  MOE  is  a  measure  of  ship  defense  capability  and 
must  be  assessed  for  each  new  ship  class.  The  PRA  As¬ 
sessment  PS&A  provides  a  roadmap  for  the  assessment 
process,  including  the  development  of  a  federation  testbed 
to  be  used  for  various  scenarios.  Both  the  PS&A,  which 
follows  the  PEO  IWS  Systems  Engineering  Process  [9] 
and  the  FEDEP  [5]  are  based  on  common  systems  engi¬ 
neering  practices,  including  the  use  of  conceptual  model¬ 
ing. 

The  Systems  Engineering  Conceptual  Model  (SECM)  is 
used  to  develop  requirements  and  to  capture  all  assump¬ 
tions,  approximations,  and  limitations  leading  from  the 
real  world  view  to  the  simulated  world  view.  This  infor¬ 
mation,  together  with  all  relevant  documentation  and  dis¬ 
cussion  captured  in  the  SECM,  support  the  VV&A  proc¬ 
ess.  The  results  from  testing  trials,  both  at-sea  and  land- 
based,  are  also  used  in  the  VV&A  process. 

The  PS&A,  including  the  SECM,  are  designed  for  reuse 
and  extension.  This  standardization  and  reuse  promotes 
risk  mitigation  and  cost  reduction  for  the  Navy’s  PRA  As¬ 
sessment  of  the  MOE  for  each  ship  class. 
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